Cognitive load driving assistant

ABSTRACT

In one embodiment, a cognitive load driving assistant increases driving safety based on cognitive loads. In operation, the cognitive load driving assistant computes a current cognitive load of a driver based on sensor data. If the current cognitive load exceeds a threshold cognitive load, then the cognitive load driving assistant modifies the driving environment to reduce the cognitive load required to perform the primary driving task and/secondary task(s), such as texting via a cellular phone. The cognitive load driving assistant may modify the driving environment indirectly via sensory feedback to the driver or directly through reducing the complexity of the primary driving task and/or secondary tasks. In particular, if the driver is exhibiting elevated cognitive loads typically associated with distracted driving, then the cognitive load driving assistant modifies the driving environment to allow the driver to devote appropriate mental resources to the primary driving task, thereby increasing driving safety.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority benefit of the U.S. ProvisionalPatent Application having Ser. No. 62/102,434 (Attorney Docket NumberHRMN/0148USL) and filed on Jan. 12, 2015. The subject matter of thisrelated application is hereby incorporated herein by reference.

BACKGROUND Field of the Various Embodiments

The various embodiments relate generally to automotive systems and, morespecifically, to a cognitive load driving assistant.

Description of the Related Art

Oftentimes, drivers fail to focus an appropriate amount of attention onthe task of driving. For example, drivers may not adjust their focus toadequately address complex driving situations attributable to traffic,the manner in which others are driving, pedestrians, road conditions,weather conditions, volume of traffic, and the like. Further, driverstypically engage in multiple, secondary in-vehicle activities thatdivert their attention from the primary task of driving. Such secondaryin-vehicle activities may include listening to loud music, participatingin conversations, texting, soothing a crying child, and so forth.

“Distracted” driving attributable to complex driving situations andsecondary in-vehicle activities increases the likelihood of collisionsand accidents. For example, a driver who is driving on a winding road atnight while talking with a passenger and operating an entertainmentsystem is more likely to become involved in an automobile accident thana driver who is focused solely on the task of driving along a straightroad during the day. Moreover, because using in-vehicle technologieswhile driving has become widespread, the frequency of injuries fromaccidents caused by distracted driving has increased. Some examples ofprevalent in-vehicle technologies are navigation systems andentertainments systems.

In general, the three primary types of driver distractions are visualdistractions, manual distractions, and cognitive distractions. Manyadverse driving conditions and in-vehicle activities lead to multipletypes of driver distractions. For example, texting is associated with avisual distraction that causes the driver to take his or her eyes offthe road, a manual distraction that causes the driver to take his or herhands off the steering wheel, and a cognitive distraction that causesthe driver to take his or her mind off the task of driving.

Because the impact of cognitive distractions on a driver is moredifficult to assess than the impact of visual distractions and manualdistractions, most drivers are oblivious to the amount of mentalresources required to perform activities and tasks. As a result, driverstypically fail to modify the driving environment or their actions toreduce their cognitive load when their level of driver distractionbecomes dangerously high.

As the foregoing illustrates, more effective techniques that enabledrivers to better understand their levels of cognitive load whiledriving would be useful.

SUMMARY

One embodiment sets forth a computer-readable storage medium includinginstructions that, when executed by a processor, cause the processor toperform the steps of computing a current cognitive load associated witha driver while the driver is operating a vehicle based on data receivedvia one or more sensors; determining that the current cognitive loadexceeds a baseline cognitive load; and in response, causing one or moreactions to occur that are intended to reduce the current cognitive loadassociated with the driver.

Further embodiments provide, among other things, a method and a systemconfigured to implement the computer-readable storage medium set forthabove.

At least one advantage of the disclosed techniques is that conveyingcognitive load levels to drivers and/or adjusting vehicle behavior basedon cognitive load levels may increase driver safety. In particular, ifthe driver is exhibiting elevated cognitive loads typically associatedwith distracted driving, then a cognitive load driving assistant maytake action to reduce the complexity of the driving situation and/or thesecondary tasks that the driver is performing, thereby allowing thedriver to devote appropriate mental resources to the primary drivingtask.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the variousembodiments can be understood in detail, a more particular description,briefly summarized above, may be had by reference to embodiments, someof which are illustrated in the appended drawings. It is to be noted,however, that the appended drawings illustrate only typical embodimentsand are therefore not to be considered limiting in scope, for thevarious embodiments may admit to other equally effective embodiments.

FIG. 1 illustrates a passenger compartment of a vehicle that isconfigured to implement one or more aspects of the various embodiments;

FIG. 2 is a more detailed illustration of the head unit of FIG. 1,according to various embodiments

FIG. 3 is a more detailed illustration of the cognitive load drivingassistant of FIG. 2, according to various embodiments;

FIG. 4 illustrates the relationship between the current driving contextand the current cognitive load of FIG. 3, according to variousembodiments;

FIG. 5 is a flow diagram of method steps for managing cognitive loadwhile driving, according to various embodiments.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toprovide a more thorough understanding of the various embodiments.However, it will be apparent to one of skill in the art that the variousembodiments may be practiced without one or more of these specificdetails.

Vehicle Overview

FIG. 1 illustrates a passenger compartment 100 of a vehicle that isconfigured to implement one or more aspects of the various embodiments.As shown, the passenger compartment 100 includes, without limitation, awindshield 110 and a head unit 130 positioned proximate to a dashboard120. In various embodiments, the passenger compartment 100 may includeany number of additional components that implement any technicallyfeasible functionality. For example and without limitation, in someembodiments the passenger compartment 100 may include a rear-viewcamera.

As shown, the head unit 130 is located in the center of the dashboard120. In various embodiments, the head unit 130 may be mounted at anylocation within the passenger compartment 100 in any technicallyfeasible fashion that does not block the windshield 110. The head unit130 may include any number and type of instrumentation and applications,and may provide any number of input and output mechanisms. For example,and without limitation, the head unit 130 typically enables the driverand/or passengers to control entertainment functionality. In someembodiments, the head unit 130 may include navigation functionalityand/or advanced driver assistance functionality designed to increasedriver safety, automate driving tasks, and the like.

The head unit 130 may support any number of input and output data typesand formats as known in the art. For example, and without limitation, insome embodiments, the head unit 130 may include built-in Bluetooth forhands-free calling and audio streaming, universal serial bus (USB)connections, speech recognition, rear-view camera inputs, video outputsfor any number and type of displays, and any number of audio outputs. Ingeneral, any number of sensors, displays, receivers, transmitters, etc.may be integrated into the head unit 130 or may be implementedexternally to the head unit 130. External devices may communicate withthe head unit 130 in any technically feasible fashion.

While driving, the driver of the vehicle is exposed to a variety ofstimuli that are related to either the primary driving task and/or anynumber of secondary tasks. For example, and without limitation, thedriver could see lane markers 142, a pedestrian 144, a cyclist 146, anda police car 148 via the windshield 110. In response, the driver couldsteer the vehicle to track the lane markers 142 while avoiding thepedestrian 144 and the cyclist 146 and apply the brake pedal to allowthe police car 148 to cross the road in front of the vehicle. Further,and without limitation, the driver could concurrently participate in aconversation 152, listen to music 154, and attempt to soothe a cryingbaby 156. Challenging driving environments and secondary activitiestypically increase the cognitive load of the driver and may contributeto an unsafe driving environment for the driver and for objects (othervehicles, the pedestrian 144, etc.) in the proximity of the vehicle. Ingeneral, the head unit 130 includes functionality to enable the driverto efficiently perform both the primary driving task and certainsecondary tasks as well as functionality designed to increase driversafety while performing such tasks.

FIG. 2 is a more detailed illustration of the head unit 130 of FIG. 1,according to various embodiments. As shown, the head unit 130 includes,without limitation, a processor 270 and a system memory 240. Theprocessor 270 and the system memory 240 may be implemented in anytechnically feasible fashion. For example, and without limitation, invarious embodiments, any combination of the processor 270 and the systemmemory 240 may be implemented as a stand-alone chip or as part of a morecomprehensive solution that is implemented as an application-specificintegrated circuit (ASIC) or a system-on-a-chip (SoC).

The processor 270 generally comprises a programmable processor thatexecutes program instructions to manipulate input data. The processor270 may include any number of processing cores, memories, and othermodules for facilitating program execution. The processor 270 mayreceive input from drivers and/or passengers of the vehicle via anynumber of user input devices 212 and generate pixels for display on thedisplay device 214. The user input devices 212 may include various typesof input devices, such as buttons, a microphone, cameras, a touch-basedinput device integrated with a display device 214 (i.e., a touchscreen), and other input devices for providing input data to the headunit 130.

The system memory 240 generally comprises storage chips such as randomaccess memory (RAM) chips that store application programs and data forprocessing by the processor 270. In various embodiments, the systemmemory 240 includes non-volatile memory such as optical drives, magneticdrives, flash drives, or other storage. In some embodiments, a storage220 may supplement or replace the system memory 240. The storage 220 mayinclude any number and type of external memories that are accessible tothe processor 170. For example, and without limitation, the storage 220may include a Secure Digital Card, an external Flash memory, a portablecompact disc read-only memory (CD-ROM), an optical storage device, amagnetic storage device, or any suitable combination of the foregoing.

As shown, the system memory 240 includes, without limitation, anentertainment subsystem 244, a navigation subsystem 246, and an advanceddriver assistance system (ADAS) 250. The entertainment subsystem 244includes software that controls any number and type of entertainmentcomponents, such as an AM/FM radio, a satellite radio, an audio andvideo computer files player (e.g., MP3 audio files player), an opticalmedia player (e.g., compact disc (CD) player), and so forth. In someembodiments, any number of entertainment components may be included inthe head unit 130 and any number of entertainment components may beimplemented as stand-alone devices. The navigation subsystem 246includes any number and type of applications that enable a driver toefficiently navigate the vehicle. For example, the navigation subsystem246 may include maps, direction routing software, and the like.

The ADAS 250 includes functionality that is designed to increase driversafety and/or automate driving tasks. For example, and withoutlimitation, in various embodiments, the ADAS 250 may provide hilldescent control, automatic parking, and the like. Notably, thefunctionality included in the ADAS 250 may supplement, enhance, and/orautomate functionality provided by other components included in thevehicle to decrease the likelihood of accidents or collisions inchallenging conditions and/or driving scenarios.

In general, challenging driving environments and distractions may strainthat ability of the driver to devote adequate attention to the primarydriving task. For example, suppose that the driver is driving thevehicle during low light conditions along a congested, windy road whiletexting on a cell phone. In such a scenario, the driver may not devoteenough mental resources to the primary driving task to operate thevehicle in a safe manner. However, many drivers do not recognize whentheir cognitive loads increase past a comfortable level and they beginto exhibit unsafe driving behaviors associated with distracted driving.For this reason, the ADAS 250 includes, without limitation, a cognitiveload driving assistant 260.

Dynamically Modifying the Driving Environment Based on Cognitive Load

In general, the cognitive load driving assistant 260 continuallyestimates the current cognitive load of the driver and determineswhether the current cognitive load indicates an abnormally stressfuldriving environment and/or an abnormal number of distractions. If thecognitive load driving assistant 260 determines that the currentcognitive load indicates an abnormally stressful driving environmentand/or an abnormal number of distractions, then the cognitive loaddriving assistant 260 attempts to indirectly or direct modify thedriving environment to reduce the cognitive load of the driver. Forexample, and without limitation, the cognitive load driving assistant260 could notify the driver of an atypically high cognitive load andsuggest alternate driving routes that are less congested than thecurrent driving route.

The cognitive load driving assistant 260 may process any type of inputdata and implement any technically feasible algorithm to estimatecurrent cognitive load and/or determine whether the current cognitiveload negatively impacts the driver's ability to safely operate thevehicle. As shown, and without limitation, the head unit 130, includingthe cognitive load driving assistant 260, receives data via any numberof driver-facing sensors 232 and non-driver-facing sensors 234. Thedriver-facing sensors 232 may include devices capable of detecting andrelaying physiological data associated with the driver. Morespecifically, the driver-facing sensors 232 may measure physiologicalchange in the body related to cognitive load. In a complementaryfashion, the non-driver-facing sensors 232 may include any devicescapable of detecting and relaying that data that does not reflect thephysiology of the driver but are related to the driving environment.

In general, the driver-facing sensors 232 and the non-driver-facingsensors 234 may include any type of sensors designed to measure anycharacteristic and may be implemented in any technically feasiblefashion. In particular, the driver-facing sensors 232 may, withoutlimitation, track specific features of the driver, such as hands,fingers, head, eye gaze, feet, facial expression, voice tone, and thelike. For example, and without limitation, the driver-facing sensors 232could include sensors that measure brain activity, heart rate, skinconductance, steering-wheel grip force, muscle activity, skin/bodytemperature, and so forth. Further, the driver-facing sensors 232 mayinclude, without limitation, microphones that detect conversationalcontext, conversational turn taking, voice tone and affect, otherauditory distractions, and the like. For example, and withoutlimitation, the driver-facing sensors 232 could detect that the driveris engaged in conversation with a passenger, the driver is currentlyspeaking, the driver's voice tone indicates that the driver is drowsy,and two other passengers are engaged in a second conversation. In someembodiments, and without limitation, the driver-facing sensors 232 mayinclude visual imagers that detect head position and orientation, facialfeatures, hands movements, etc. In some embodiments, and withoutlimitation, the driving facing sensors 232 may include depth sensorsthat detect finger and hand gestures, body posture, and as forth and/oreye gaze and pupil size tracking sensors.

In a complementary fashion, and without limitation, thenon-driver-facing sensors 234 may track any features of the vehicleand/or environment surrounding the vehicle that are relevant to thedriver. For example, and without limitation, the non-driver-facingsensors 234 may track vehicle control elements, such as the position ofthe throttle, the position of the clutch, gear selection, the locationof the brake pedal, the angle of the steering wheel, and so forth. Thenon-driver-facing sensors 234 may include any number of sensors fortracking vehicle speed, position, orientation, and dynamics, such asinertial and magnetic sensors. Further, the non-driver-facing sensors234 may include devices that detect and/or track stationary and/ormoving objects surrounding the vehicle. Such detection sensors mayinclude, without limitation, a front-mounted visible light imager, aninfrared imager, a radio detection and ranging (RADAR) sensor, a lightdetection and ranging (LIDAR) sensor, a dedicated short rangecommunication (DSRC) sensor, thermal and motion sensors, depth sensors,sonar and acoustic sensors, and the like. In some embodiments, andwithout limitation, the non-driver-facing sensors 234 may include remotesensors that provide information regarding local weather, traffic, etc.

The driver-facing sensors 232 and the non-driver-facing sensors 234 maybe deployed in any technically feasible fashion. For example, andwithout limitation, the driver-facing sensors 232 and thenon-driver-facing sensors 234 may include any number and combination ofvehicle-integrated sensors, vehicle-integrated imagers, wearable devices(affixed to or worn by the driver), and remote sensors. In one example,and without limitation the, driver-facing sensors 232 could includesteering wheel-mounted sensors that measure heart rate, skinconductance, and grip force, while the non-driver facing sensors 234could include a front-mounted visible light imager, an infrared imager,and a LIDAR sensor.

In some embodiments, the cognitive load driving assistant 260 mayreceive additional input data, referred to herein as advanced driverassistance system (ADAS) data. Such ADAS data may include, withoutlimitation, data received from a global navigation satellite system(GNSS) receiver 236, data received from the navigation subsystem 245,and data received from the entertainment subsystem 244. The globalnavigation satellite system (GNSS) receiver 236 determines globalposition of the vehicle. The GNSS receiver 236 operates based on one ormore of the global positioning system of manmade Earth satellites,various electromagnetic spectrum signals (such as cellular towersignals, wireless internet signals, and the like), or other signals ormeasurements, and/or on a combination of the above items. In variousembodiments, the cognitive load driving assistant 260 accesses globalpositioning data from GNSS receiver 236 in order to determine a currentlocation of the vehicle. Further, in some embodiments, the cognitiveload driving assistant 260 accesses data provided by the navigationsubsystem 246 in order to determine a likely future location of thevehicle. In some embodiments, the cognitive load driving assistant 260accesses data provided by entertainment subsystem 244 to assess theimpact of secondary tasks, such as listening to music, on the cognitiveload of the driver.

In yet other embodiments, the cognitive load driving assistant 260 mayreceive and transmit additional ADAS data including, and withoutlimitation, automotive vehicle-to-everything (V2X) data 238. Thevehicle-to-everything (V2X) data 238 may include vehicle-to-vehicle(V2V) data, vehicle-to-infrastructure (V2I) data, and so forth. The V2Xdata 238 enables the vehicle to communicate with other objects thatinclude V2X capabilities. For example, the vehicle may communicate withother vehicles, smartphones, traffic lights, laptops, road-side V2Xunits, and so forth.

After receiving the input data, the cognitive load driving assistant 260computes any number of cognitive metrics that relate to the currentcognitive load of the driver. Subsequently, the cognitive load drivingassistant 260 determines whether the cognitive metrics indicate that thedriver may be unable to devote a typical and/or safe amount of mentalresources to the primary task of driving. In general, the cognitive loaddriving assistant 260 may compute any number of cognitive metrics andassess whether the cognitive metrics indicate an elevated currentcognitive load in any technically feasible fashion. For example, andwithout limitation, for a subset of the driver-facing sensors 232, thecognitive load driving assistant 260 could compute a current value for acognitive metric and compare the current value to historical values forthe cognitive metric. Substantially in parallel, for each of theremaining driver-facing sensors 232, the cognitive load drivingassistant 260 could compare current sensor data to historical sensordata. The cognitive load driving assistant 260 could then determinewhether the results of the various comparisons indicate an elevatedcurrent cognitive load.

For example, and without limitation, the cognitive load drivingassistant 260 could compute a weighted average of the deviations of thevalues of any number of cognitive metrics and any number ofdriver-facing sensors 232 from historical values to determine an averagedeviation. If the average deviation exceeds a certain preset limit, thenthe cognitive load driving assistant 260 could determine that thecurrent cognitive load is elevated. In another example, the cognitiveload driving assistant 260 could compare the value of a primarycognitive load metric to historical values of the primary cognitive loadmetric to determine whether the current cognitive load may be elevated.Additionally, the cognitive load driving assistant 260 could compare thevalues of any number of driver-facing sensors 232 to historical valuesto provide a confidence measurement.

In general, the cognitive load driving assistant 260 may compute acurrent cognitive load based on any number, including one, of cognitivemetrics and sensor data. Further, the cognitive load driving assistant260 may determine historical values for cognitive metrics, cognitiveloads, and/or sensor data in any technically feasible fashion. Forexample, and without limitation, in some embodiments the cognitive loaddriving assistant 260 may store the current cognitive load and otherrelevant data, referred to herein as a “driving context” in anyavailable memory (e.g., the system memory 240). The driving context mayinclude any number and type of data such as time of day, the location ofthe vehicle, detailed sensor readings, and so forth. Subsequently, thecognitive load driving assistant 260 may retrieve previously storedcognitive loads and driving contexts to determine historical cognitiveloads at any level of situational granularity. For example and withoutlimitation, in some embodiments, the cognitive load driving assistant260 may compute an average cognitive load based on all historicalcognitive loads. In other embodiments, and without limitation, thecognitive load driving assistant 260 may compute an average cognitiveload based on the historical cognitive loads in similar driving contexts(e.g., the same time of day and/or location).

In some embodiments and without limitation, the cognitive load drivingassistant 260 may transmit and/or receive cognitive loads and,optionally, driving contexts to other a cognitive load database 282 thatis included in a cloud 280 (e.g., encapsulated shared resources,software, data, etc.). The cognitive load driving assistant 260 andother cognitive load driving assistants included in other vehicles maythen retrieve information from the cognitive load database 282. Thecognitive load driving assistant 260 may analyze such data as part ofevaluating the current cognitive load, detecting situations that involvehigh cognitive loads, and so forth.

In some embodiments, the cognitive load driving assistant 260 maytransmit and/or receive cognitive loads and, optionally, drivingcontexts with other cognitive load driving assistants 260 as V2X data238. In general, the cognitive load driving assistant 260 may beconfigured to transmit and store data relevant to the cognitive load ofthe driver in any technically feasible fashion. Similarly, the cognitiveload driving assistant 260 may be configured to receive and process datarelevant to the cognitive loads of other drivers as well as anyadditional factors that may influence the cognitive load of the otherdrivers in any technically feasible fashion.

After determining the current cognitive load of the driver and assessingother relevant data, the cognitive load driving assistant 260 mayperform any number of actions designed to increase the safety of thedriver. As previously detailed, such relevant data may include, withoutlimitation, such current location of the vehicle, time of day, dataprovided by the navigation subsystem 246 and the entertainment subsystem244, cognitive loads of drivers along the planned driving route, and soforth. The actions may directly or indirectly modify the driving taskand any secondary tasks that may distract the driver.

For example, and without limitation, the cognitive load drivingassistant 260 could provide feedback to the driver via the displaydevice 214. The feedback could include the current cognitive load,historical cognitive loads, and suggestions for reducing the complexityof the primary driving task, such as easier (less congested) drivingroutes or lanes. In some embodiments, and without limitation, thecognitive load driving assistant 260 may reduce human machine interface(HMI) complexity to reduce distractions. For example, and withoutlimitation, the cognitive load driving assistant 260 could blockincoming cellular phone calls, lower the volume of music, blocknon-critical alerts (e.g., low windshield washer fluid alert, etc.), andthe like.

In some embodiments, the cognitive load driving assistant 260 mayperform actions designed to preemptively increase driving safety. Forexample, and without limitation, suppose that the cognitive load drivingassistant 260 detects elevated cognitive loads associated with otherdrivers in the proximately of the vehicle or along the driving routespecified by the navigation subsystem 246. To increase the vigilance ofthe driver, the cognitive load driving assistant 260 may alert thedriver to expect potentially hazardous situations (e.g., accidents,dangerous curves, etc.) and/or distracted drivers.

In some embodiments and without limitation, the cognitive load drivingassistant 260 may work in conjunction with the navigation subsystem 246and/or other elements included in the ADAS 250 to increase drivingsafety based on one or more predictive heuristics. In some embodiments,the cognitive load driving assistant 260 could configure the navigationsubsystem 246 to avoid locations associated with elevated cognitiveloads. For example, and without limitation, if elevated historicalcognitive loads are associated with a particular exit to an airport,then the cognitive load driving assistant 260 could configure thenavigation subsystem 246 to preferentially select an alternative exit tothe airport. In other embodiments, upon detecting elevated cognitiveloads of the driver or nearby drivers, the cognitive load drivingassistant 260 could modify one or more ADAS parameters to increase theconservatism of the ADAS 250. For example, and without limitation, thecognitive load driving assistant 260 could configure preemptive brakingto activate at an earlier time or could decrease the baseline at whichthe ADAS 250 notifies the driver of a lane departure from the currentdriving lane.

The cognitive load driving assistant 260 may configure the vehicle toprovide feedback to the driver in any technically feasible fashion. Forexample, and without limitation, the cognitive load driving assistant260 may configure the vehicle to provide any combination of visualfeedback, auditory feedback, haptic vibrational feedback, tactilefeedback, force feedback, proprioceptive sensory feedback, and so forth.Further, the cognitive load driving assistant 260 may configure anyfeatures of the vehicle in any technically feasible fashion. Forexample, the cognitive load driving assistant 260 may configure theentertainment subsystem 244, the navigation subsystem 246, applicationsincluded in the ADAS 250, and any control mechanisms provided by thevehicle via any number of control signals or via any type of interface.

As described above, in some embodiments the cognitive load drivingassistant 260 receives cognitive load data and/or related data fromother vehicles (e.g., via the cognitive load database 282, the V2X data282, etc.). In operation, the cognitive load driving assistant 260 mayleverage such shared data in any technically feasible fashion tooptimize driving safety either at the current time or at a future time.For example, and without limitation, instead of comparing the currentcognitive load to a personalized average cognitive load, the cognitiveload driving assistant 260 could compare the current cognitive load to abaseline cognitive load based on collective cognitive loads of manydrivers normalized for time, location, and other factors. In general,the cognitive load driving assistant 260 attempts to maintain thecurrent cognitive load below the threshold represented by the baselinecognitive load.

In another example, and without limitation, the cognitive load drivingassistant 260 may examine the average cognitive load of drivers in closeproximity to the vehicle or along a driving route associated with thevehicle to detect a preponderance of elevated cognitive loads thatindicates a complex situation, such as an accident. Upon detecting suchan area of elevated cognitive loads, the cognitive load drivingassistant 260 may generate a sensory warning designed to cause thedriver to become more vigilant, generate a new driving route that avoidsareas of elevated cognitive load, and so forth. In yet another example,and without limitation, the cognitive load driving assistant 260 maygenerate a “heat map” based on collective cognitive loads. The cognitiveload driving assistant 260 may then suggest altering the drivingenvironment based on the heat map. In particular, the cognitive loaddriving assistant 260 may recommend lane changes to lanes associatedwith lower cognitive loads; interact with the navigation subsystem 246to optimize the driving route, and the like.

In general, the cognitive load driving assistant 260 may be configuredto process any type of input data and/or compute any number of metricsrelated to cognitive load. Further, the cognitive load driving assistant260 may be configured to increase driving safety and/or improve thedriving experience based on the processed data and metrics in anytechnically feasible fashion. Although the cognitive load drivingassistant 260 is described in the context of the head unit 130 herein,the functionality included in cognitive load driving assistant 260 maybe implemented in any technically feasible fashion and in anycombination of software and hardware. For example, and withoutlimitation, each of the processor 270 and the system memory 240 may beembedded in or mounted on a laptop, a tablet, a smartphone, asmartwatch, a smart wearable, or the like that implements the cognitiveload driving assistant 260. In other embodiments, and withoutlimitation, the cognitive load driving assistant 260 may be implementedas a stand-alone unit that supplements the functionality of existingvehicle safety systems. Such a stand-alone unit may be implemented as asoftware application that executes on any processor.

FIG. 3 is a more detailed illustration of the cognitive load drivingassistant 260 of FIG. 2, according to various embodiments. As shown, thecognitive load driving assistant 260 includes, without limitation, apupillometery engine 320, a body state engine 330, a cognitive loadanalyzer 340, a current driving context 370, and a cognitive loadfeedback engine 380. In alternate embodiments and without limitation,any number of components may provide the functionality included in thecognitive load driving assistant 260 and each of the components may beimplemented in software, hardware, or any combination of software andhardware.

In operation, the pupillometry engine 320 receives pupil data from apupil sensor 302 that measures the sizes of the driver's pupils via eyetracking tools. Based on the pupil data, the pupillometry engine 320computes a pupil-based metric that reflects the cognitive load of thedriver. The pupillometry engine 320 may compute the pupil-based metricin any technically feasible fashion. For example, and withoutlimitation, the pupilloemetry engine 320 may analyze the pupil data toidentify specific rapid changes in pupil size that are associated withincreased cognitive load.

Operating substantially in parallel to the pupillometry engine 320, thebody state engine 330 receives sensor data from a heart rate sensor 304,a galvanic skin response (GSR) sensor 306, and a blood pressure (BP)sensor 308. Based on the sensor data, the body state engine 330 computesa body-based metric that reflects the cognitive load of the driver. Thebody state engine 330 may compute the body-based metric in anytechnically feasible fashion. For example, and without limitation, thebody state engine 330 may evaluate the heart rate in conjunction withthe skin rate to determine a level of psychophysiological arousal.Further, the body state engine 330 may evaluate the BP to estimate anamount of blood flow in the front part of the brain. In general, thebody state engine 330 may evaluate any type of sensor data in anycombination to compute any number of metrics that reflect the cognitiveload of the driver.

As shown, the cognitive load analyzer 340 receives the pupil-basedmetric and the body-based metric and computes a current cognitive load350 that approximates the cognitive load of the driver. The cognitiveload analyzer 340 may compute the current cognitive load 350 in anytechnically feasible fashion. For example, and without limitation, thecognitive load analyzer 340 may compute the current cognitive load 350as a weighted average of the pupil-based metric and the body-basedmetric. In various embodiments, the cognitive load analyzer 340 mayperform any number of comparison operations between the current value ofany number of metrics and any number and type of corresponding baselinevalues to determine the current cognitive load 350. Further, thecognitive load analyzer 340 may determine that the value of a particularmetric is erroneous based on the values of other metrics. In someembodiments, the cognitive load analyzer 340 may compute the currentcognitive load 350 based on a subset of metrics and compute a confidencevalue based on a different subset of metrics.

While the cognitive load driving assistant 260 evaluates data receivedvia the driver-facing sensors 232, the cognitive load driving assistant260 also generates a current driving context 370 that includes datareceived via the non-driver-facing sensors 234, data received via theGNSS receiver 236, and the V2X data 238. The current driving context 370described the current driving environment. As shown, the current drivingcontext 370 includes, without limitation, driving task parameters 372,secondary task parameters 378, vehicle parameters 374, and environmentalparameters 376. In general, the driving task parameters 374 directlyinfluence a driving task load that represents the mental resourcesrequired to perform the primary driving task. By contrast, the secondarytask parameters 460 directly influence a secondary task load thatrepresents the mental resources required to perform secondary tasks,such as operating the entertainment subsystem 244 or talking on acellular phone. The vehicle parameters 374 and the environmentalparameters 376 reflect circumstances that impact the mental resourcesrequired to perform the driving task and/or the secondary tasks. Forexample, and without limitation, the vehicle parameters 374 and theenvironmental parameters 376 could include the location of the vehicle,the condition of the road, the weather, the lighting conditions, and soforth.

As shown, the cognitive load feedback engine 380 receives the currentcognitive load 350 and the current driving context 370 and generates,without limitation, feedback signals 388, driving adjustment signals382, entertainment subsystem adjustment signals 384, and navigationsubsystem adjustment signals 386. In operation, the cognitive loadfeedback engine 380 evaluates the current cognitive load 350 relative toa baseline cognitive load to determine whether the current cognitiveload 350 is elevated. The cognitive load feedback engine 380 maydetermine the baseline cognitive load in any technically feasiblefashion. For example, and without limitation, the baseline cognitiveload could be a predetermined constant value. In some embodiments, thecognitive load feedback engine 380 may dynamically compute the baselinecognitive load based on any number and type of historical dataassociated with any number of drivers and any number of drivingcontexts.

If the cognitive load feedback engine 380 determines that the currentcognitive load 350 is elevated relative to the baseline cognitive load,then the cognitive load feedback engine 380 may endeavor to reduce thecurrent cognitive load 350. Notably, the cognitive load feedback engine380 may examine the current driving context 370 to determine how tooptimize the driving environment to reduce the driving task load and/orthe secondary tasks loads. In general, the cognitive load feedbackengine 380 may generate any number of control signals in any technicallyfeasible fashion that is consistent with the capabilities and interfacesimplemented in the vehicle. Such control signals may provide, withoutlimitation, any combination of visual feedback, auditory feedback,haptic vibrational feedback, tactile feedback, force feedback,proprioceptive sensory feedback, and so forth.

For example, and without limitation, the cognitive load feedback engine380 could transmit the feedback signals 388 that configure the displaydevice 214 to provide visual feedback regarding the current cognitiveload 350, historical cognitive loads, and recommendations for reducingthe driving task and/or secondary tasks loads. If the vehicle isequipped with the advanced driving features, then the cognitive loadfeedback engine 380 could increase the conservatism of the vehicle viathe driving adjustment signals 382, such as decreasing a baseline atwhich the ADAS 250 notifies the driver of a lane departure. In someembodiments, the cognitive load feedback engine 380 may configure theentertainment subsystem 244 via the entertainment subsystem adjustmentsignals 384 to reduce distractions associated with an in-vehicle audiosystem. In yet other embodiments, the cognitive load feedback engine 380may configure the navigation subsystem 246 via the navigation subsystemadjustment signals 386 to replace a current driving route with a newdriving route that is less congested, thereby lowering the mentalresources required to perform the primary driving task.

FIG. 4 illustrates the relationship between the current driving context340 and the current cognitive load 350 of FIG. 3, according to variousembodiments. As shown, the current driving context 340 includes thedriving task parameters 372, the secondary task parameters 378, thevehicle parameters 374, and the environmental parameters 376. Ingeneral, the driving task parameters 372 directly influence a drivingtask load 450 that represents the mental resources required to performthe primary driving task. By contrast, the secondary task parameters 460directly influence a secondary task load 460 that represents the mentalresources required to perform secondary tasks, such as talking on a cellphone.

Together, the driving task parameters 372, secondary task parameters378, vehicle parameters 374, and environmental parameters 376 contributeto the current cognitive load 350. In particular, as the driving taskload 450 and/or the secondary task load 460 increases, the currentcognitive load 350 increases (depicted as an increasing cognitive load472) within an overall cognitive load 470. The overall cognitive load470 represents the total cognitive load of the driver and, within theoverall cognitive load 470, a baseline cognitive load 474 reflects thetypical cognitive loads of the driver.

As shown, initially the current cognitive load 350 exceeds the baselinecognitive load 474. In response, the cognitive load feedback engine 380analyzes the current driving context 370 and transmits the navigationsubsystem adjustment signal 386 “reroute via less congested roads” tothe navigation subsystem 246, and the entertainment subsystem adjustmentsignal 384 “mute the audio system” to the entertainment subsystem 244.Subsequently, as a result of the reduction in the driving task load 450and the secondary task load 460 attributable to, respectively, thenavigation subsystem adjustment signal 386 and the entertainmentsubsystem adjustment signal 384, the current cognitive load 350decreases and no longer exceeds the baseline cognitive load 474.

As the foregoing example illustrates, in general, if the currentcognitive load 350 exceeds the baseline cognitive load 474, then thecognitive load feedback engine 380 attempts to adjust the currentdriving context 340 to either directly or indirectly reduce the currentcognitive load 350. Accordingly, the level of driver distraction isreduced and the safety of the driver and surrounding drivers isincreased.

FIG. 5 is a flow diagram of method steps for managing cognitive loadwhile driving, according to various embodiments. Although the methodsteps are described in conjunction with the systems of FIGS. 1-4,persons skilled in the art will understand that any system configured toimplement the method steps, in any order, falls within the scope of thevarious embodiments.

As shown, a method 500 begins at step 504, where the cognitive loaddriving assistant 260 included in a vehicle receives sensor data via thedriver-facing sensors 232 and the non-driver-facing sensors 234. Thedriver-facing sensors 232 may include any number of sensors that monitorcharacteristics of the driver. For example and without limitation, thedriver-facing sensors 232 may include the pupil sensor 302, the heartrate sensor 304, the galvanic skin response (GSR) sensor 306, the bloodpressure (BP) sensor 308, and the like. By contrast, thenon-driver-facing sensors 324 monitor data that is not directly relatedto the driver, such as environmental data and vehicle data.

At step 506, the cognitive load driving assistant 260 computes thecurrent cognitive load 350 based on the driver-facing sensor data. Atstep 508, the cognitive load driving assistant 260 computes the currentdriving context 370 based on the non-driver-facing sensor data inconjunction with other relevant environmental and vehicle data. Theadditional data may include any type of data received in any technicallyfeasible fashion. For example, and without limitation, the additionaldata could include a location of the vehicle based on data received viathe GNSS receiver 236 and locations of other vehicles based on V2X data238. As persons skilled in the art will recognize, the cognitive loaddriving assistant 260 typically performs steps 506 and steps 508substantially in parallel.

At step 510, the cognitive load driving assistant 260 transmits thecurrent cognitive load 350 and the current driving context 370 to thecognitive load database 282 included in the cloud 280. Sharing cognitivedata in this manner enables other cognitive load driving assistants 260included in other vehicles to alert other drivers when the currentcognitive load 350 indicates that the driver of the vehicle may pose asafety risk.

At step 512, the cognitive load feedback engine 380 computes thebaseline cognitive load 474 based on historical cognitive load data inconjunction with historical driving contexts. The historical cognitiveload data and the historical driving contexts may be stored in anymemory, in any technically feasible fashion, and include any amount ofdata associated with any number of drivers. For example, and withoutlimitation, the historical cognitive load data could be stored in thecognitive load database 282 and include data for many drivers. Thecognitive load feedback engine 380 may compute the baseline cognitiveload 474 in any technically feasible fashion. For example, and withoutlimitation, the cognitive load feedback engine 380 could compute thebaseline cognitive load 474 as the average of all historical cognitiveloads associated with the driver.

At step 514, the cognitive load feedback engine 380 compares the currentcognitive load 350 to the baseline cognitive load 474. If, at step 514,the cognitive load feedback engine 380 determines that the currentcognitive load 350 is not greater than the baseline cognitive load 474,then the method 500 returns to step 504 where the cognitive load drivingassistant 260 receives new sensor data. If, however, at step 514, thecognitive load feedback engine 380 determines that the current cognitiveload 350 is greater than the baseline cognitive load 474, then themethod 500 proceeds directly to step 516.

At step 516, the cognitive load feedback engine 380 provides feedback tothe driver indicating the elevated current cognitive load 350. Thecognitive load feedback engine 380 may provide the feedback in anytechnically feasible fashion and may include any additional data forreference. For example, and without limitation, the cognitive loadfeedback engine 380 could display an “evaluated cognitive load” warningvia the dashboard-mounted display device 214. The warning could includethe current cognitive load 350 and an indication of how the currentcognitive load 350 relates to the baseline cognitive load 474. Inanother example, and without limitation, the cognitive load feedbackengine 380 could audibly warn the driver that the current cognitive load350 indicates a dangerous driving situation.

At step 518, the cognitive load feedback engine 380 performs correctiveactions designed to reduce the driving task load 450 and/or thesecondary task load 460 based on the current driving context 370 and/orthe historical driving contexts. For example, and without limitation,the cognitive load feedback engine 380 could determine that the currentdriving route is challenging and, in response, interact with thenavigation subsystem 246 to suggest a less congested route for thevehicle. In another example, and without limitation, the cognitive loadfeedback engine 380 could determine that the number of secondary tasksthat the driver is performing significantly exceeds the number ofsecondary tasks that the driver typically performs and, in response,interact with the entertainment subsystem 244 to mute the speakers.

The method 500 then returns to step 504 where the cognitive load drivingassistant 260 receives new sensor data. The cognitive load drivingassistant 260 continues to cycle through steps 504-518, assessing thecurrent cognitive load 350 to detect and attempt to minimize situationsassociated with elevated cognitive loads until the vehicle or thecognitive load driving assistant 260 is turned off.

In one embodiment, a cognitive driving assistant analyzes driver-facingsensor data and provides feedback regarding elevated driver cognitiveloads to enable drivers to recognize and react to dangerous drivingenvironments. In operation, the cognitive driving assistant processesdriver-facing sensor data to compute a current cognitive load.Substantially in parallel, the cognitive driving assistant processes nondriver-facing sensor data along with other relevant data, such as GNSSdata, to generate a current driving context. The current driving contextincludes driving parameters, vehicle parameters, environmentalparameters, and secondary task parameters.

Because the impacts of different “distractions,” such as talking on acellular phone, vary between individual drivers, a cognitive loadfeedback engine analyzes the current cognitive load of the driver withrespect to historical cognitive loads of the driver in similar drivingcontexts. For example, if a current time included in the current drivingcontext indicates night time lighting conditions, then the cognitiveload feedback engine could compare the current cognitive load of thedriver to historical cognitive loads in other driving contexts thatindicate night time lighting conditions. If the cognitive load feedbackengine determines that the current cognitive load is greater than the“baseline” cognitive load in similar driving contexts, then thecognitive load feedback engine initiates corrective action. Thecorrective action may include any type of passive feedback, such as anaudible warning, or any type of active control, such as disabling aringer of a cellular phone.

In some embodiments, the cognitive load feedback engine transmits thecurrent cognitive load and/or the current driving context to a cognitiveload database stored in a public cloud. Such information enables othercognitive load feedback engines operating in other vehicles topreemptively identify dangerous driving situations. For example, if thecurrent cognitive load of the driver is elevated, then a cognitive loadfeedback engine in a second vehicle located in the immediate vicinity ofthe vehicle could notify the driver of the second vehicle that adistracted driver is nearby.

At least one advantage of the disclosed approach is that because thecognitive load feedback engine enables drivers to adjust driving and/orsecondary task behavior based on cognitive loads, driver safety may beincreased. In particular, educating drivers on their cognitive loadlevels and/or the cognitive load levels of nearby drivers providesdrivers with an opportunity to increase their concentration on theprimary driving task during challenging driving situations and/or reducetheir concentration on secondary tasks. Consequently, driver safety maybe increased for the driver as well as nearby drivers.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments.

Aspects of the present embodiments may be embodied as a system, methodor computer program product. Accordingly, aspects of the presentdisclosure may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “circuit,”“module” or “system.” Furthermore, aspects of the present disclosure maytake the form of a computer program product embodied in one or morecomputer readable medium(s) having computer readable program codeembodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, enable the implementation of the functions/acts specified inthe flowchart and/or block diagram block or blocks. Such processors maybe, without limitation, general purpose processors, special-purposeprocessors, application-specific processors, or field-programmableprocessors or gate arrays.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While the preceding is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. A computer-readable storage medium includinginstructions that, when executed by a processor, cause the processor toperform the steps of: computing a current cognitive load associated witha driver while the driver is operating a vehicle based on data receivedvia one or more sensors; determining that the current cognitive loadexceeds a threshold cognitive load; and in response, causing one or moreactions to occur that are intended to reduce the current cognitive loadassociated with the driver.
 2. The computer-readable storage medium ofclaim 1, wherein causing one or more actions to occur comprisesconfiguring a display screen to display the current cognitive load. 3.The computer-readable storage medium of claim 1, wherein causing one ormore actions to occur comprises configuring an entertainment subsystemto reduce the volume of an audio signal.
 4. The computer-readablestorage medium of claim 1, wherein causing one or more actions to occurcomprises causing a navigation subsystem to generate a modified drivingroute that has less cognitive complexity than a current driving route.5. The computer-readable storage medium of claim 1, wherein the one ormore sensors measure one or more physiological changes in a body of thedriver.
 6. The computer-readable storage medium of claim 5, wherein theone or more physiological changes are associated with at least one of abrain activity, a heart rate, a skin conductance, and sizes of pupils.7. The computer-readable storage medium of claim 5, wherein computingthe current cognitive load comprises: performing a multiplicationoperation between a first measurement received via a first sensorincluded in the one or more sensors and a first weight to generate afirst weighted measurement; performing a multiplication operationbetween a second measurement received via a second sensor included inthe one or more sensors and a second weight to generate a secondweighted measurement; and computing an average of the first weightedmeasurement and the second weighted measurement.
 8. Thecomputer-readable storage medium of claim 1, further comprisingcomputing a confidence factor associated with the current cognitive loadbased on data received via one or more other sensors.
 9. Thecomputer-readable storage medium of claim 8, wherein at least one or theone or more sensors measures sizes of pupils and the one or more othersensors measure at least one of a brain activity, a heart rate, and askin conductance.
 10. The computer-readable storage medium of claim 1,further comprising computing the threshold cognitive load based on aplurality of previous cognitive loads computed for the driver.
 11. Thecomputer-readable storage medium of claim 1, further comprisingcomputing the threshold cognitive load based on a plurality of currentcognitive loads computed for a plurality of other drivers.
 12. A methodfor sharing cognitive load while driving, the method comprising:computing a first cognitive load associated with a first driver based ondata received via one or more sensors, wherein the first driver and theone or more sensors are associated with a first vehicle; and sharing thefirst cognitive load with a second driver that is associated with asecond vehicle.
 13. The method of claim 12, wherein sharing the firstcognitive load comprises storing the first cognitive load in a cognitiveload database included in a cloud that is accessible to the secondvehicle.
 14. The method of claim 12, wherein sharing the first cognitiveload comprises transmitting vehicle-to-everything data that includes thefirst cognitive load to the second vehicle.
 15. The method of claim 12,further comprising: receiving a second cognitive load that is associatedwith the second driver; and in response, modifying a characteristic ofthe first vehicle.
 16. A system configured to manage cognitive loadwhile driving, the system comprising: a memory storing a cognitive loaddriving assistant; and a processor that is coupled to the memory and,when executing the cognitive load driving assistant, is configured to:compute a current cognitive load associated with a driver, while thedriver is operating a vehicle, based on data received via one or moresensors; determine that the current cognitive load exceeds a thresholdcognitive load; and taking one or more actions that are intended toreduce the current cognitive load below the threshold cognitive load.17. The system of claim 16, wherein taking one or more actions comprisesconfiguring a display screen to display the current cognitive load. 18.The system of claim 16, wherein taking one or more actions comprisesconfiguring an advanced driver assistance system (ADAS) to increase theconservatism of one or more safety features associated with the vehicle.19. The system of claim 16, wherein the processor is further configuredto: compute a current driving context based on data received via one ormore other sensors; select a previous driving context from a cognitiveload database based on the current driving context; and assign thethreshold cognitive load based on a previous cognitive load associatedwith the previous driving context.
 20. The system of claim 16, whereinthe processor is further configured to compute a confidence factorassociated with the current cognitive load based on data received viaone or more other sensors.